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Convergence Time Analysis of Quantized 
Gossip Consensus on Digraphs 

Kai Cai and Hideaki Ishii 
Abstract 

We have recently proposed quantized gossip algorithms which solve the consensus and averaging 
problems on directed graphs with the least restrictive connectivity requirements. In this paper we study 
the convergence time of these algorithms. To this end, we investigate the shrinking time of the smallest 
interval that contains all states for the consensus algorithm, and the decay time of a suitable Lyapunov 
function for the averaging algorithm. The investigation leads us to characterizing the convergence time 
by the hitting time in certain special Markov chains. We simplify the structures of state transition by 
considering the special case of complete graphs, where every edge can be activated with an equal 
probability, and derive polynomial upper bounds on convergence time. 

I. Introduction 

Inspired by aggregate behavior of animal groups and motion coordination of distributed robotic net- 
works, the consensus problem has been extensively studied in the recent literature of systems control 
(e.g., [l]-[3]). The objective of consensus is to have a population of nodes, each possessing an initial 
state, agree eventually on some common value through only local information exchange. This problem is 
also intimately related to oscillator synchronization [4], load balancing [5], and leader election [6]. The 
averaging problem is of a special form, with the goal to decentrally compute the average of all initial 
states at every node. 

We have recently proposed in [7], [8] randomized gossip algorithms which solve the consensus and 
averaging problems on directed graphs (or digraphs), under a quantization constraint that each node has 
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an integer-valued state. In particular, our derived connectivity condition ensuring average consensus is 
weaker than those in the literature [2], in the sense that it does not postulate balanced topologies. Here 
the main difficulty is that the state sum of nodes cannot be preserved during algorithm iterations. This 
scenario was previously considered in [9], [10], where averaging is guaranteed in expectation but there 
is in general an error in mean square and with probability one. By contrast, we overcome this difficulty 
by augmenting the so-called "surplus" variables for individual nodes so as to maintain local records of 
state updates, thereby ensuring average consensus almost surely. 

In this paper and its conference precursor [11], we investigate the performance of our proposed 
algorithms by providing upper bounds on the mean convergence time. The state transition structures 
resulting from these algorithms turn out to be rather complicated. Hence in our analysis on convergence 
time, we focus on the special case of complete graphs. The analysis is still challenging, but we will 
also discuss that the general approach can be useful for other graph topologies. First, for the consensus 
algorithm, we find that the mean convergence time is 0(n 2 ). To derive this bound, we view reaching 
consensus as the smallest interval containing all states shrinking its length to zero. This perspective leads 
us to characterizing convergence time by the hitting time in a certain Markov chain, which yields the 
polynomial bound. Second, we obtain that the mean convergence time of the averaging algorithm is 
0(n 3 ). As the original algorithm in [7], [8] is found to induce complex state transition structures, we 
have suitably revised it to manage the complexity. For the modified algorithm, a Lyapunov function is 
proposed which measures the distance from average consensus. We then bound convergence time by 
way of bounding the number of iterations required to decrease the Lyapunov function; the latter is again 
characterized by the hitting time in a special Markov chain. 

Our work is related to [12]— [15], which deal also with the convergence time of gossip averaging 
algorithms with quantized states. In [12], a Lyapunov approach is adopted and polynomial bounds on 
convergence time are obtained for fully connected and linear networks. The work [13] generalizes these 
bounds to arbitrarily connected networks (fixed or switching), utilizing the results on the meeting time of 
two random walks on graphs. Also, bounds for arbitrarily connected networks are provided in [14], [15]; 
these bounds are, however, in terms of graph topology rather than the number of nodes. In these cited 
references, a common feature is that the graphs are undirected. By contrast, our algorithm in [7], [8] is 
designed for arbitrary strongly connected digraphs, and we are interested in studying the corresponding 
convergence time. 

To bound the convergence time, a frequently employed approach is to bound the decay time of 
some suitable Lyapunov functions [12], [16]. In particular, [16] derives tight polynomial bounds on the 
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convergence time of synchronized averaging algorithms, with either real or quantized states. In addition, 
[17] investigates a variety of quantization effects on averaging algorithms, and demonstrate favorable 
convergence properties by simulations. Our work adopts the Lyapunov method, as in [12], [16]; the 
common function used in these papers turns out, however, not to be a valid Lyapunov function for our 
averaging algorithm. This is due again to that the state sum does not remain invariant, and the augmented 
surplus evolution must also be taken into account. According to these features, we establish an appropriate 
Lyapunov function, and prove that bounding its decay time can be reduced to finding the hitting time in 
a certain Markov chain. 

A. Setup and Organization 

Consider a digraph Q = (V, £), where V = {1, n} is the node set, and £ C V x V the edge set. Each 
directed edge (j, i) in £ , pointing from j to i, denotes that agent j communicates to agent i (namely, 
the information flow is from j to i). Selfloop edges are not allowed, i.e., £. Communication 

among the nodes is by means of gossip: At each time instant, exactly one edge (j, i) G £ is activated 
independently from all earlier instants and with a time-invariant positive probability pji G (0, 1) such 

that E(j,i)eePji = L 

To model the quantization effect in information flow, we assume that at time k G Z + (nonnegative 
integers), each node has an integer-valued state Xi(k) G Z, i € V; the aggregate state is denoted by 

x(k) = [xi(fc) • • • x n (k)] T G Z n . Let 

X :={x:m<Xi<M, i G V}, (1) 

for some (finite) constants m, M. Suppose throughout the paper that the initial state satisfies x(0) G X. 
Also, let 1 = [1 • • • 1] T be the vector of all ones. 

For the convergence time analysis, we will impose the following two assumptions on the graph topology 
and the probability distribution of activating edges. Let | • | denote the cardinality of a set. 

Assumption 1. The digraph Q is complete (i.e., every node is connected to every other node by a directed 
edge). It follows that there are \£\ = n(n — 1) edges. 

Assumption 2. The probability distribution on edge activation is uniform; namely, each edge can be 
activated with the same probability p := l/\£\. 

The rest of this paper is organized as follows. In Section II, we formulate and solve the problem 
of convergence time analysis for the consensus algorithm. Then in Sections III and IV, we derive an 
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upper bound for the convergence time of the averaging algorithm. Further, we compare convergence rates 
through a numerical example in Section V, and finally we state our conclusions in Section VI. 

II. Convergence Time of Quantized Consensus Algorithm 
A. Problem Formulation 

First we recall the quantized consensus (QC) algorithm from [7]. Suppose that the edge (j, i) € £ is 
randomly activated at time k. Along the edge node j sends to i its state information, Xj(k), but does 
not perform any update, i.e., Xj(k + 1) = xj(k). On the other hand, node % receives fs state xj(k) and 
updates its own as follows: 

(Rl) If Xi(k) = xj(k), then Xi(k + 1) = Xj(fe); 

(R2) if Xi(k) < Xj(k), then X{(k + 1) € (xi(k),Xj(k)]; 

(R3) if Xi(k) > Xj(k), then X{(k + 1) € [xj(k),Xi(k)). 

Let the subset of IT 1 be the set of general consensus states: 

& := {x : xi = ■■■ = x n }. (2) 

We say that the nodes achieve general consensus almost surely if for every initial state x(0), there exist 
T < oo and x* <G ^ such that x(k) = x* for all k > T with probability one. Under QC algorithm, a 
necessary and sufficient graphical condition that ensures almost sure general consensus is that the digraph 
Q contains a globally reachable node (i.e., a node that is connected to every other node via a directed 
path) [7]. Clearly if Q is complete, then every node is globally reachable. 

The convergence time of QC algorithm is the random variable T qc defined by T qc := mf{k > : 
x(k) € ( tf}. The mean convergence time (with respect to the probability distribution on edge activation), 
starting from a state xq G X, is then given by 

E qc (x ) :=E[T, c |a;(0) = x ] . (3) 

Problem 1. Let Assumptions 1 and 2 hold. Find an upper bound of the mean convergence time E qc (xo) 
of QC algorithm with respect to all possible initial states xq € X. 

We now present the main result of this section: an upper bound of the mean convergence time E qc (xo) 
for all possible initial states xo G X. 

Theorem 1. Let Assumptions 1 and 2 hold. Then 

max£ gc (xo) < n(n - 1)(M - m) = 0(n 2 ). 

To derive this bound, we first provide preliminaries on the hitting time in finite Markov chains. 
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B. Preliminaries on Hitting Time 

Let {Xk}k>o be a Markov chain with a finite state space S and a transition probability matrix P = (Py) 
(e.g., [18]). The entry Py denotes the one-step transition probability from state i to state j. In particular, 
the diagonal entry Pa denotes the selfloop transition probability. A state i € S is said to be absorbing if 
Pa = 1. For a given {X k } k >o, the hitting time of a subset T of 5 is the random variable H? ({X k } k >o) 
defined by 

Hr ({X k } k > ) := inf{Z >0:I,eT}. 

The mean time (with respect to the probability distribution specified by P) taken for the chain, starting 
from a state i G S, to hit T is given by 

oo 

P, := E [H T ({X k } k > ) \X = i]=J2 l - ¥ [ R r ({X k } k >o) = l\X = i] , (4) 

1=0 

where E[-|-] and P[-|-] denote the conditional expectation and conditional probability operators, respec- 
tively. Here is an important fact on mean hitting times [18, Theorem 1.3.5]. 

Lemma 1. The vector of mean hitting times (Pj)j 6 5 of a subset T satisfies the system of linear equations 

Ei = for i € T, 

Using Lemma 1, we derive a closed-form expression of the mean hitting time for a specific Markov 
chain; this chain will be shown to characterize the state transition structure under QC algorithm. For the 
proof of this result, see Appendix. 

Lemma 2. Consider the Markov chain in Fig. 1 with transition probabilities 

Vz + r z + q z = 1, Vz = Qz {z = 1, ...,n - 1), r = 1, r n = 1. 
Then the mean hitting time of the state or n starting from state z is 

2-1 . n-1 

^ , 2, \ ^ i z \ — v n — 7 , 

^ = 1 - - E - + - E — - ( z = n - 1 ■ 

1=1 ]=Z J 

C. Analysis of Convergence Time 

We now proceed as follows. For an arbitrary x(k) define the minimum and maximum states by 

m(k) := minxi(k), M{k) := maxxj(fc). (5) 
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Fig. 1. Markov chain I: states and n are absorbing. Here ro, . . . , r n are selfloop transition probabilities. 

We view the state x(k) converging to <?f as the interval [m(k),M(k)\ shrinking to length 0. Let the 
random variable T qc be the time when one interval shrinkage occurs; then the corresponding mean time, 
starting from a state x, is E qc (x) := E [T„ c |x G X\. Since one shrinkage decreases the interval length 
by at least 1, there can be at most M — m shrinkages for xo G X. It then follows that 

max E qc (xq) < max E^ c (x) ■ (M — m). (6) 

x a <=X xt=X 

Consider a subset X\ of X defined by 

X\ := {x : x\ = ■ ■ ■ = x z = 1 Hz x z+ \ = ■ ■ ■ = x n = zG [1, n — 1]}. (7) 

Thus the interval has length 1 for all x G X\. It is not difficult to see that max^g^ E qc (xo) = 
max X £x Eq C (x). The following lemma states an upper bound of E qc (xo) for xo G 

Lemma 3. Let Assumptions 1 and 2 hold. Then max Io6 ^ E qc (xo) < n(n — 1) = 0(n 2 ). 

Proof. By Assumptions 1 and 2, every directed edge in Q can be activated with the uniform probability 
p = l/(n(n — 1)). Starting from an arbitrary state in the set X\, the transition structure under QC 
algorithm is the Markov chain displayed in Fig. 1; in the diagram, 

state : the vector = [0 • • • 0] T of all zeros, 
< state n : the vector 1 = [1 • • • 1] T of all ones, (8) 

z 

state z : the vector \\ ■ ■ ■ 1 • • • 0l T in X\, 
and the transition probabilities are p z = q z = z(n — z)p, z G [1, n — 1]. To see this, consider the transition 
from state z to state z + 1; this occurs when an edge is activated, with xj = 1 and xi = 0, 

so that (R2) of QC algorithm applies. Since there are z(n — z) such edges, the transition probability 
p z = z(n — z)p. Likewise, one may derive that the transition from state z to state z — 1 is with probability 
q z = z{n — z)p, which occurs when (R3) of QC algorithm applies. Now observe in Fig. 1 that the states 
0, n G ^ and 1, ...,n — 1 G X\\ hence max 2e [ ll! _ 1 ] E z = max Io£ ^ E qc (xq), where E z is from (4). 
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It is left to invoke the formula of E z in Lemma 2 for the obtained transition probabilities, which yields 

z—l ^ n—l ^ 

£i = (i-£)V i +£ri 

n ^— ' (n — i p n ^-^ in 

i=l y n j=z J ^ 



^ 2 z — 1 z n — z 

n (n — z + \)p n zp 

n — z 1 1 , 

< - = n(n — lj. 



n — z + 1 p p 

Thus E z < n(n — 1) for all z G [l,n — 1]. Therefore max^g^ E qc (x ) < n(n — 1) = 0(n 2 ). ■ 
Finally, our main result (Theorem 1) on upper bounding E qc (xo) for xq £ X follows immediately 
from Lemma 3 and (6). 

Remark 1. We discuss the idea of how this result for complete graphs might be extended to handle more 
general topologies. We still view reaching consensus as the interval [m(k), M(k)] shrinking to length 
0; thereby the inequality (6) holds. We then again consider the subset X\ given in (7), and as long as 
the digraph is strongly connected (i.e., every node is connected to every other node) one can verify 
that the state transition structure under QC algorithm is still the one in Fig. 1. The associated transition 
probabilities, however, depend crucially on topologies. In order to apply again Lemma 2 to derive bounds, 
it would be important to establish the relation between transition probabilities and graph topologies; this 
will be targeted in our future work. 

III. Quantized Averaging Algorithm and Its Lyapunov Function 

In this and next sections, we address the convergence time analysis for the quantized averaging (QA) 
algorithm, which is a modification of the one in [7], [8]. We start by presenting the modified algorithm, 
and formulate the corresponding time analysis problem. We then propose a Lyapunov function, which 
turns out to be a suitable measure for the average consensus error. In Section IV, we will derive an upper 
bound on the mean convergence time by means of bounding the decay time of the proposed Lyapunov 
function. 

A. Problem Formulation 

First we present QA algorithm. As in [7], [8], since the state sum cannot be preserved at each time 
instant, we associate each node i 6 V with an additional surplus variable, Si(k) G Z, to locally record 
the changes of Xi(k). The aggregate surplus is denoted by s(k) = [si(k) ■ ■ ■ s n (k)] T G Z™, whose initial 
value is set to be s(0) = [0 • • • 0] T . Now suppose that the edge (j, i) G £ is activated at time k. There 
are two stages: (I) Along the edge, node j sends to % its state Xj(k) and surplus Sj(k). Node j does not 



I 1 x j{k) I 1 

(xi(k),Si(k)) nodei- node j (xj{k),0) 
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Fig. 2. Stage (I): Node j sends to i its state and surplus through the edge (j, i). 



(0 (x'm,m) | node i 



(ii) (xj(fc), Sj(fc)) node i 



node i 




node j 








node j 




node j 









Fig. 3. Stage (II): Either (i) node i updates its state and surplus, or (ii) it sends Sj(k) back to node j through edge 



update its state, but sets its surplus to be after transmission (see Fig. 2). (II) Based on the information 
received, node i determines either to update its state and surplus, or to send back to j the surplus Sj(k) by 
activating the opposite edge (see Fig. 3). Notice that the latter operation in (II) requires bidirectional 
communication between two nodes at a single time instant; this is possible in complete digraphs (our 
assumption), but not in general strongly connected digraphs. 

Formally, QA algorithm is described as follows. 
(Rl) If xi(k) = Xj(k), then there are two cases: 

(i) If Si(k) > & Sj(k) > 0, then 

Xi(k + 1) = Xi(k), Si(k + 1) = Si(k); 
Xj(k + 1) = Xj(k), Sj(k + 1) = Sj(k). 

(ii) Otherwise (i.e., either surplus equals zero), 

Xi (k + 1) = Xi(k), Si (k + 1) = Si(k) + 8j (k) G {0, 1}; 
xj(k + 1) = xj(k), sj(k + 1) = 0. 

(R2) If xi(k) < Xj(k), then there are two cases: 
(i) If Si{k) + Sj(k) > 0, then 

Xi(k + 1) = xt(k) + 1, Si(k + 1) = Si(k) + Sj(k) — 1 € {0, 1}; 
Xj (k + 1) = xj(k), sj(k + 1) = 0. 
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(ii) Otherwise (i.e., Sj(fc) + Sj(k) = 0), 

Xi(k + 1) = Xi(k), Si(k + 1) = Si(k) + Sj(k) = 0; 
Xj (k + 1) = Xj(k), sj(k + 1) = 0. 

(R3) If Xi(k) > Xj(k), then there are two cases: 

(i) If Si(k) + Sj(k) = 0, then 

Xi(k + 1) = Xi(k) - 1, si(fc + 1) = Si (A;) + Sj(k) + 1 = 1; 
Xj (k + 1) = Xj(k), sj(k + 1) = 0. 

(ii) Otherwise (i.e., Si(k) + Sj(k) > 0), 

Xi(k + 1) = Xi(k), Si(k + 1) = Si(k); 
Xj(k + 1) = Xj(k), Sj(k + 1) = Sj(k). 

In the algorithm, observe that (1) (Rl)(i) and (R3)(ii) are where node i sends Sj(k) back to node j 
in stage (II), which requires bidirectional communication; (2) only (R3)(i) 'generates' one surplus, and 
only (R2)(i) 'consumes' one surplus; (3) the quantity (x + s) T l stays invariant, i.e., for every k > 0, 

{x(k + 1) + s(k + 1)) T 1 = (x(k) + s(k)) T l = x(0) T l. (9) 

Distinct from the algorithm in [7], [8], this QA algorithm does not involve the threshold constant and 
the local extrema variables, thus reducing individual computation effort. Also each surplus variable is 
indeed binary-valued, and therefore requires merely one bit for both storage and transmission. A further 
difference between the two algorithms lies in the use of surplus variables: The algorithm in [7], [8] 
allows surpluses to pile up, which is indeed required to achieve average consensus for arbitrary strongly 
connected digraphs. By contrast, our QA algorithm here prevents surpluses from piling up, and meanwhile 
simplifies the transition structure. In addition, unlike the algorithm in [12] which assumes bidirectional 
communication for all time, the design of surplus updates here marks a feature of our QA algorithm. 
Now let the subset srf of Z n x Z n be the set of average consensus states: 

si := {(x,s) : Xi = Lx(0) T l/nJ or [x(0) T l/n], i G V}. (10) 

We say that the nodes achieve average consensus almost surely if for every initial condition (x(0),0), 
there exist T < oo and (x*,s*) € £f such that (x(k),s(k)) = (x*,s*) for all k > T with probability 
one. Here is the convergence result of QA algorithm for complete digraphs. 
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Proposition 1. Let Assumption 1 hold. Then, under QA algorithm, the nodes achieve average consensus 
almost surely. 

This convergence result may be justified by a similar argument as given in [7], [8]; some care, however, 
has to be taken for the operations on surplus variables, as pointed out above. For completeness, the proof 
is provided in the Appendix. In addition, we note that the convergence can also be implied by the time 
analysis using Lyapunov approach in Section IV below. 

The convergence time of QA algorithm is the random variable T qa defined by T qa := inf{£; > 
: (x(k), s(k)) <G The mean time taken for this convergence (according again to the probability 
distribution on edge activation), starting from (xo,0) with xo £ X, is then given by 

E qa (x ) :=E[T, o |(x(0),0) = (x ,0)]. (11) 

Problem 2. Let Assumptions 1 and 2 hold. Find an upper bound of the mean convergence time E qa (xo) 
of QA algorithm with respect to all possible initial states xo € X. 

Our main result is the following upper bound of E qa (xo) with respect to all possible initial states 

xo € X. 

Theorem 2. Let Assumptions 1 and 2 hold. Then 

n . , 2/ x 3(M-m) , .R(R-1) n , 
max E qa ( Xo ) < n\n - l)-^ + *(* " V^rJT) = °^ 

where R € [0, n — 1] is an integer, as in (12) below. 

We note that the order of this polynomial bound is the same as that in [12] for undirected, complete 
graphs. To derive this bound, we will first propose a valid Lyapunov function for QA algorithm. Then 
we will upper bound the mean convergence time by way of upper bounding the mean decay time of the 
Lyapunov function. 

B. Lyapunov Function 

We start by introducing two variables, called positive surplus S+ and negative surplus S*_; they are 
global variables, but are needed only for the convergence time analysis. Write the initial state sum 

x(0) T l = nL + R, (12) 

where L := [x(0) T l/n\ is one of the possible values for average consensus, and < R < n. Observe 
that when a surplus is generated/consumed, the corresponding state moves one-step either closer to or 
farther from the value L. Positive and negative surplus variables are used to identify these two directions. 
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Concretely, when a surplus is generated, we increase S + (resp. S-) if the corresponding state moves 
towards (resp. away from) L. On the other hand, when a surplus is consumed, we distinguish the following 
two situations: In one case where the state moves closer to L, we decrease 5_ if it is nonzero, and S + 
otherwise; in the other case where the state moves away from L, we decrease only S + . 

We now formalize the updating rules of S+ and S-. Let D(k) := Y17=i ~~ L\ be the sum of 

average consensus errors, and suppose that the edge (j, i) € £ is activated at time k. 

(51) If (R3)(i) generates one surplus, then there are two cases: 

(i) If D(k + 1) = D(k) - 1 (i.e., Xi{k) > L), then 

S + {k + l) = S+{k) + l. 

(ii) If D(k + 1) = D{k) + 1 (i.e., Xi (k) < L), then 

S-(k + l) = S-(k) + 1. 

(52) If (R2)(i) consumes one surplus, then there are also two cases: 

(i) If D(k + 1) = D(k) + 1 (i.e., Xi (k) > L), then 

S+{k + l) = S + (k)-l. 

(ii) If D{k + 1) = D{k) - 1 (i.e., Xi {k) < L), then 

S_(k) = => S + (k + 1) = S+(k) - 1; 
S-(k) > S^(k + 1) = S^(k) - 1. 

(53) Otherwise 

S+(fc + l) = S+(k); 
S-(k + l) = S-(k). 

The case (S3) above includes (Rl), (R2)(ii), and (R3)(ii) of QA algorithm; note that, in these cases, 
there is no state update. Since initially there is no surplus in the system (i.e., s(0) = 0), we set <S+(0) = 
S-(0) = 0. Also, one may readily see that S + (k) + S-(k) = s(k) T l, which relates the global surpluses 
to the local ones. 

We are ready to define the Lyapunov function V(k), k > 0, which is given by 

V(k) := D(k) + S+(k) - S-(k). (13) 
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It is not difficult to see from (S1)-(S3) that V(k) is non-increasing. Indeed, V(k) stays put except for only 
one case - (S2)(ii) and negative surplus S-(k) = - where it decreases by 2, i.e., V(k + 1) = V(k) — 2. 
Notice that after this decrement, S + (k + 1) > and S-(k + 1) = 0. 

Remark 2. We emphasize that the validity of V(k) as a Lyapunov function is not restricted only to 
undirected graphs, since the updating rules (S2) and (S3) do not involve (Rl)(i) and (R3)(ii) where 
bidirectional communication is required. Indeed, V(k) is a suitable Lyapunov function for the original 
QA algorithm in [7], [8], which can achieve average consensus on arbitrary strongly connected digraphs. 
This is one contribution of our work, which might also provide a preliminary to attack convergence time 
on more general topologies. 

In the following lemma, we collect several useful implications from the definition of function V(k). 

Lemma 4. 

(1) A lower bound of V(k) is R, i.e., V(k) > R for all k. 

(2) If V{k) = R, then S-{k) = 0, S+(k) > 0, and (Vi G [l,n]) Xi (k) > L. 

(3) If D(k) = 0, then S-(k) = and V(k) = S + (k) = R. 

(4) Suppose R = 0. Then D(k) = if and only if V(k) = 0, and in both cases SL(jfe) = S+(k) = 0. 
Proof. We prove these statements in this order: (2), (1), (3), and (4). 

(2) Let V(k) = R. Then there must exist k < k such that V(k - 1) = R + 2 and V(k ) = 
R. Also we have S+(/co) > and S-(ko) = 0. Now assume Xi(ko) < L. It follows from (9) that 
x\(ko) + Y17=2 x i{ko) + s(ko) T l = nL + R. Rearranging the terms and by s(ko) T l = S+(ko) + S-(ko), 
we obtain Y^ =2 Xi{ko) — (n — 1)L = (L — xi(ko)) + R — S + (ko). Thus 

n 

V(k ) = (L — xi(fco)) + Xi(ko) + S+(k ) - S-(k ) 

= 2(L-x 1 (k )) +R > R. 

This contradicts V(ko) = R, and hence Xi(ko) > L for all i. The latter holds also for time k because 
the minimum states are non-decreasing by QA algorithm. Finally, according to the updating rules of S+ 
and S-, one may easily see that S-(k) = and S + (k) > 0. 

(1) When V(k) = R, every state Xi(k) > L and consequently (S3)(ii) cannot occur. As V(k) is 
non-increasing, it is lower bounded by R. 

(3) Let D(k) = 0. Then x(kfl = nL, and thus S+(k) + S-(k) = s(kfl = R. It follows that 
V(k) = S+{k) - S-(k) < R. But V(k) > R, so that necessarily V(k) = S+(k) - S-(k) = R, which 
also implies that S-(k) = and S + (k) = R. 
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(4) Assume R = 0. (Only if) The conclusion follows immediately from (3). (If) Let V(k) = 0. Then 
there must exist k$ < k such that V(ko — 1) = 2 and V(ko) = 0. Also we have S+(ko) > and 
S-(ko) = 0. Hence D(ko) + S + (ko) = 0, which results in D(k ) = S + (k ) = 0. As average consensus 
is achieved at k$, no further state or surplus update occurs. So the conclusion for time k follows. ■ 

Next, we find an upper bound for the function V(k). 

Proposition 2. Let x(0) G X in (1). Then for every k > 0, 

V(k)< {M - 2 m)n +R. 

Proof. Since the function V(k) is non-increasing, it suffices to find an upper bound for V(0) = 
Ya=i l x «(0) ~~ L\- Consider the function V(0) — R; it is convex in x(0), and X is a convex set. Hence, 
one of the extreme points of X is a maximizer. Fix r G [l,n], and let x(0) G X be such that xi(0) = 
••• = x r (0) = m and x r+ i(0) = ••• = x n (0) = M. Then V(0)-R = r(L - m) + (n - r)(M - L) - R. 
Also we have L = (l T x(0) — R)/n = (rm + (n — r)M — R)/n. Substituting this into the above equation 
and rearranging the terms, we derive 

1^(0) -R= J-^L-^r- + ( 2( M - m) - 2*\ r 

2(M - m) 



n 



i. r ,y i. r 

r (n — — ) + —In 



2 V M-m A M-m' 



2(M-m) 1, R ,o , w o . 

< - ■ i (n - ( equality holds iff r = \{n- ^) ) 

_ 1 (n(M - m) - R) 2 
~ 2 n(M - m) 

\{n{M-m)f (M-m)n 
^ o — 777 \ — = o ( equality holds iff R = ). 

2 n{M — m) 2 

Thus V(A;) — R is upper bounded by (M — m)n/2, which is achievable if and only if R = and r = n/2. 



IV. Convergence Time Analysis of QA Algorithm 

We turn now to analyzing the mean convergence time of QA algorithm, by way of upper bounding 
the mean decay time of the Lyapunov function V(-) in (13). This Lyapunov approach is also adopted 
in [12], [16]; the common function used is V'(k) = X^=i( x «(^) — x(0) T l/n) 2 . It can be verified that 
V'(k) is, however, not a valid Lyapunov function with respect to our QA algorithm. This is due to that 
the state sum is not preserved in each iteration and the surplus evolution must also be taken into account, 
as in our function V(k). 
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<J2 <J3 <74 qn-2 Qn-1 



Fig. 4. Markov chain II: state n is absorbing. 




Fig. 5. Markov chain III: state n is absorbing. 



A. Preliminaries on Hitting Time 

As in Subsection II-B, we provide preliminaries on the hitting time in finite Markov chains, specific 
to the analysis of QA algorithm. For the proofs see Appendix. 

Lemma 5. Consider the Markov chain in Fig. 4 with transition probabilities 

pi + ri = l, p z + r z + q z = 1 (z = 2, ...,n - 1), r n = l. 
Then the mean hitting times of the state re starting from state 1 and z are respectively 



n-l 



1=2 

n-l 



n 

\i=2 



i= 



Pi / pi 



i 



+£ n 

j=2 \i=i+l 



1 

Pi I Pj 



1 



+ 



Pi 



\i=2 



Pi / Pi 



j=2 \i=j 



(z = 2,...,n-l). 



Lemma 6. Consider the Markov chain in Fig. 5 with transition probabilities 

pi + ri + di = 1, Pz + r z + q z + d z = I {z = 2, n - 2), 

r n _i + g n _i + d n __i = 1, p n -i + r^- + q n ~i + ^n-i = 1, = 1- 



Here • and • denote the states of the lower and upper rows, respectively. Then for states re — 1 and re — 1, 
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V(k) 



no) 

V(O) - 2 
V(O) - 4 



Fig. 6. Decay of function V(k) in case R = 



their mean hitting times of the absorbing state n are 

(n-1 \ n n-1 I n-1 \ 
n|)^g(,n|)i. 

In the rest of this section, the proof of Theorem 2 is given. We will need the following notation. Define 
the random variable Ty := ini{k > : V(k) = R}; thus Ty is the time when V(-) decreases to R. The 
mean decay time, starting from {xq, 0), is then given by 

E v (x Q ) :=E[Ty|(x(0),0) = (x ,0)]. (14) 

Now recall R from equation (12); we proceed with two cases in this order: R = and R > 0. When 
R = the mean convergence time E qa (xo) is found to satisfy E qa (xo) = Ey(xo), whereas when R > 
we have E qa (xo) > Ey(xo) in general and the corresponding analysis turns out to be based on the 
former case. 



B. Proof for the case R = 

In this case, the mean convergence time E qa (xo) is characterized by the mean time that the function 
V(k) decays to 0; that is, E qa {xo) = Ey(xo) in (14). This is because by Lemma 4 (4), V(k) = if and 
only if D(k) = 0, and the latter implies (x(k), s(k)) G ■ As each decrement reduces V{k) by 2, the 
initial value V(0) is necessarily even, and there need in total V(0)/2 decrements. 

To upper bound Ey(xo), we view the decay of V(k) as the descent of level sets in the (n + 2)- 
dimensional space of the triples u := (x, S + , 5_) (see Fig. 7). In this space, the average consensus state 
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Fig. 7. Decay of V(k) viewed as level set descent in the (n + 2) dimensions of (x, S+, S-). Descending is possible only 
from the shaded area and through the dotted curves. 



is simply the point (LI, 0,0). Define the level sets 

n 

Ui := {u : V = \xi - L\ + S + - S- = 2 • I}, 1 = 1, V(0)/2. 

i=i 

Thus when u(k) G Ui, we interpret that (x(k), s(k)) is /-step away from (i.e., V(k) requires I 
decrements to reach 0). Also, it is important to note that on every level set Ui, the triple evolution may 

° 

Uf :={ueUr. S- = k S + > 0}. 



start, and may descend to the next level, only from a strict subset U? defined by 



To see this, first recall that the decrement of V(-) (i.e., level set descent) requires S- = and S + > 0. 
Moreover, for the outmost level Wy( )/2> tne initial triple is of the form (xq, 0, 0); and for each subsequent 
level, the triple evolution starts right after descending from the preceding level, where we have S- = 
and S + > 0. 

Now let the random variable T\ be the time of one decrement of V(-). The corresponding mean time, 
starting from a triple u G Uf, is then given by E[(u) := E [Ti\u G Uf] , I G [l,y(0)/2]. Since the 
initial value V(0) is upper bounded by (M — m)n/2 (Proposition 2), the function V(-) requires at most 
(M — m)n/4 decrements to reach 0. Hence, an upper bound of its mean decay time is the following: 



/ / \ (M — m)n 

maxE v (x )< max E ](u) ■ — — . (15) 

x ex ie[i,v(o)/2],ueu? 4 



Here is a key result. 
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Proposition 3. Let Assumptions 1 and 2 hold. Then 



max E\ (u) < 6n(n - 1) = 0(n 2 ). 
le[i,V(p)/2],ueU° 

To prove Proposition 3, it suffices to establish 

max£}(u) < 6n(n - 1) = 0(n 2 ), (16) 

for every I G [1, V(0)/2]. In the sequel we will provide the proof for the case I = 1 (i.e., one step away 
from average consensus), which contains the essential idea of our argument. Specifically, we first exhaust 
the possible triple evolution under QA algorithm, second derive the evolution structure and transition 
probabilities, and third calculate the corresponding mean hitting time. The analysis of the case I > 2 
follows in a similar fashion but is more involved; we refer to Appendix for the proof. 

Proof for the case 1 = 1: Without loss of generality let L = 1. We investigate the triple evolution from 
the level set U\, starting in U\, to the average consensus state (1,0,0). By Assumptions 1 and 2, every 
directed edge in Q can be activated with the uniform probability p = l/(n(n — 1)). Consider the triple 

n-2 

([2 1 • • • 1 0] , 0, 0) G U®; we show that either S- or S+ can be generated. Case 1: an edge (J,i) 
is activated, with Xj = and xi = 1. In this case, (R3)(i) of QA algorithm applies, and the resulting 

n— 3 

triple is ([2 1 • • • 1 0] T , 0, 1) £ 14\ — Uf. There are n — 2 such edges; so the probability of this 
transition is (n — 2)p. In fact, such transitions can continue until all the ones become zeros, generating 
in total S- = n — 2. Case 2: an edge (J, i) is activated, with Xj = or 1 and x, = 2. Again (R3)(i) 

n-l 

of QA algorithm applies, the resulting triple being (fl 1 - • 1 0] T , 1, 0) E lA®. This transition is with 
probability (n — l)p, since there are n — 1 such edges. 
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n-1 



Now starting from the triple (fl 1 • • • 1 0] , 1, 0), on one hand, we can have a similar process, as from 

n-2 

([2 1 • • • 1 0] T , 0, 0) described above, generating in total SL = n — 2. On the other hand, observe that 
there is only one edge (j, i) such that xj = 1, Sj = 1, and x« = 0, S{ = 0. If this edge is activated (with 
probability p), then (R2)(i) of QA algorithm applies, and the resulting triple is the average consensus 
state (1,0,0). 

Based on the above descriptions, we derive that the transition structure from U\ to (1, 0, 0) under QA 
algorithm is the one displayed in Fig. 8. 1 In this diagram, the state n is the average consensus state 
(1,0,0), and the other states belong to U\, listed below: 



n-1 : ([2 1 1 ••• 1 1 0] T , 0, 0) 
n-2 : ([2 1 1 ••• 1 0] T , 0, 1) 



2 : ([2 10 
1 : ([2 



0] T , 0, n-3) 
0] T , 0, n-2) 



n-1 : ([111 
~^~2 : ([111 

2 : ([110 
T : ([10 



1 1 0] T , 1, 0) 
1 0] T , 1, 1) 

0] T , 1, n-3) 
0] T , 1, n-2) 



Note that negative surplus is zero (£_ = 0) only in the states n — 1 and n — 1; hence these two triples 
are in 14®. Also, one may verify that the transition probabilities are as follows: 

p 1 = (n- 2)p, di = p; p n -i = p, q n ^i = (n - 2)p, d n _i = (n - 1>; 

p z = (n - 1 - z)zp, q z = (z-l)p, d z = zp (z = 2, n - 2). 

To upper bound E\ (u) for u G U\, in Fig. 8 we add transitions from the state ztoz with the probability d z , 
z G [1, n— 1], thereby increasing the probabilities of moving away from the average consensus state n. This 
modification leads us to the same structure displayed in Fig. 5; thus, we have max uGW o E\{u) < E n _\, 
where E n _\ is given in (4). 

It is left to calculate E n _\ with respect to the obtained transition probabilities. For this we invoke the 
formulas in Lemma 6. First, 

n' 1 qi n — 2 n — 3 n — 4 2 1 1 

2 p~ ~ 1 ' n-2 ' 2{n - 3) ' ' ' (n - 4)3 ' (n - 3)2 ~ (n-3)!' 

'The transition structure in Fig. 8 is obtained with a minor modification from the original. For those triples in Ui —U°, we 
treat the following transitions from left to right as selfloops: For some node i such that Xi = and Si — 0, its state Xi increases 
by consuming one negative surplus (under R2(i) of QA algorithm). By treating such transitions as selfloops, only the probability 
of moving towards the average consensus state is reduced; so it can be verified that the mean hitting time derived from this 
structure is an upper bound of that from the original. We make such modifications in our analysis henceforth. 
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Similarly, 

n-1 




2 

_i_ . . . _i_ ( n — 2) • I 

(n-3)! (n-2)p ' (n-4)! (n - 3)2p y ' (n - 2)p p 



2 



1 1 

+ 7 ^7 + -" + l + l 



(n-2)! (n-3)! 



P 

< - 3 = 6n(n - 1) = 0(n 2 ). 
P 

Finally, E^-i < (1 + (p„-iR-i)) £— = (1 + (p/((n - l)p))) • 6n(n - 1) < 6n(n - 1) = 0(n 2 ). ■ 
Therefore, it follows from Proposition 3 and equation (15) that the upper bound of E qa (xo) in 
Theorem 2 holds for the case R = 0. 

C. Proof for the case Re [l,n — 1] 

When i? ^ 0, we have -E ga (xo) > £V(xo) i n general. This is because F(/c) = R does not generally 
imply (x(k), s(k)) G and even after reaches its lower bound R (Lemma 4 (1) and (2)), the 

pair (x(k), s(k)) may require extra time to reach srf . Define the level set Ur := {u : V = J2i=i \ x i ~ 
L\ + S + — S- = R}; then the mean convergence time starting from a triple u € Ur is given by 
E qa (u) := E [T qa \u € Ur]. Also recall from (14) that Ey(xo), with xo G X in (1), denotes the mean 
decay time of V(k) to the lower bound R. From these we obtain the mean convergence time of QA 
algorithm 

max E qa (xo) < max Ey{xo) + max E qa (u). (17) 

x a ex x ex u&a r 

In the sequel, we find upper bounds for Ey{xo) and E qa (uR), respectively. First, as in the case R = 0, 
we have 

max^y(xo) <n 2 (n-l) 3(M ~ m) = 0(n 3 ). (18) 

Xo&X 2 

This is due to the following reason. The function V(k) decays from its initial value V(0) to R, and 
each decrement reduces V(k) by 2. It follows that V(0) — R is necessarily even and there need in total 
(V(0) — R)/2 decrements. For I G [1, (V(0) — R)/2] recall that E\{u) denotes the mean time spent for 
one decrement of V(k), starting from a triple u G Uf. Following Proposition 3, one may similarly derive 
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M(k) 

L + R 
L + R-l 

L + 2 
L + l 



Fig. 9. Decrement of maximum state when u £ Ur 



that maxj 6 h (y(o)-.R)/2],«eW ^i( u ) < 6n(n — 1). Moreover, V(0) — i? < (M — m)n/2 by Proposition 2; 
thus requires at most (M — m)n/4 decrements to reach it!. Therefore, max I[)g ^ £V(£o) < 

maXi e[li(v(0) __ R)/2]jU6W o • (M - m)n/4 < n 2 (n - 1)3(M - m)/2 = 0(n 3 ). 

Next, we find an upper bound for max ue ^ R E qa (u). By Lemma 4 (2) we have (Vz G V) Xj > L; so 
the maximum state M(k) in (5) satisfies M(fc) G [L,L + R\. If R = 1, then in fact (ic(fc),s(&)) G <fi/; 
thus in this case E qa {uR) = 0, and we have from (17) and (18) that max^g,^ E ga (xo) = 0(n 3 ). It is left 
to consider R G [2, n — 1]. Since M(k) = L or L + 1 implies (x(/c), s(k)) G =2^, the mean convergence 
time E qa {u) can be characterized by the mean time that M(k) decays to L + 1. The decay of M(k) is 
displayed in Fig. 9; observe that M(/c) requires at most R — 1 decrements to reach L + l. Let Em{u) 
denote the mean time taken for one decrement of M(k), starting from a triple u G Ur. Then an upper 
bound for E qa (u) is as follows: 

max E qa (u) < m&x Em(u) ■ (R — 1). (19) 
Proposition 4. Let Assumptions 1 and 2 hold. Then 

R 

m&xE M (u) < n(n-l) — — = 0(n 2 ). 

u&A R n — (R/2) 

To prove Proposition 4, we first find the subset in which one decay of M(k) takes the longest time, 

second derive the transition structure and probabilities under QA algorithm, and third compute the mean 

hitting time. 

Proof of Proposition 4. We consider the following two cases when R is even and odd, respectively. 
1) R is even. Let U e be a subset of Ur given by U e := {u = (x, S+,S~) : x G X e , S + = S- = 0}, 
where 

X e := {x : x\ = ■ ■ ■ = xr = L + 2, xr +1 = ■ ■ ■ = x n = L}. 
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For a state in X e , one decrement of its maximum value L + 2 occurs only when all the R/2 state 
components having that value decrease; thus it is not hard to see max ue u R Em(u) = max ue ^ e Em(u). 

Now pick an arbitrary triple u in U e ; we investigate its evolution under QA algorithm. If an edge 
(j, i) is activated, with Xj = L and xi = L + 2, then (R3)(i) of QA algorithm applies, and the resulting 

(R/2)-l 



triple is ((L + 2 ■ ■■L + 2 L+IL ■ ■ ■ L] T , 1, 0). Namely, one maximum state decreases. Also observe 
that there are (R/2) (n — (R/2)) such edges; so the probability of this transition is (R/2) (n — (R/2))p, 
where p = l/(n(n — 1)) by Assumptions 1 and 2. Indeed, this process can continue until all the R/2 
maximum states decrease to the value L + 1, and we derive that the corresponding transition structure 
under QA algorithm is the one displayed in Fig. 4 with the length n = (R/2) + 1. In the diagram, 

R/2 

state 1 : ({L + 2 L + 2 ■■ ■ L + 2 L + 2 L ■■■ L] T , 0, 0) 

state 2: ([L + 2L + 2- --L + 2L+1L-- - L] T , 1, 0) 

state R/2 : ([L + 2 L + 1 L + 1 • • • L + 1 L ■ ■ ■ L] T , (R/2) - 1, 0) 

state (R/2) + 1: ([L+1L+1L + 1 ••• L + U ••■ L] T , R/2, 0) 
and the transition probabilities are pi = (R/2)(n- (R/2))p, p z = ((R/2) - z + l)(n - (R/2))p, 
q z = ( z - l)((R/2) -z+ l)p, z G [2, R/2}. Observe that the state 1 eU R and the state (R/2) + 1 G A; 
so max uG £4 Em(u) = E\, where E\ is from (4). 

It remains to invoke the formulas in Lemma 5 to calculate E\. First, 

^j? ft (R/2) - 1 ((R/2) - 2)2 2((R/2) - 2) (R/2) - 1 



=2 



Pi n-(R/2) 2(n-(R/2)) ((R/2) - 2)(n - (R/2)) ((R/2) - l)(n - (R/2)) 



am 



i)\ { (r/2) - l y*™- 1 

R/2)-l ~ [ n -(J?/0\ i ^ i5 



(n - (R/2))W 2 )- 1 ~ \n - (R/2) J 
the last inequality is due to R < n. Similarly \\f^ 2 q-i/pi < 1 for i = 3, ...,R/2. Then we obtain 

(S ft ) ' ft + § ft ) ' S < (« " W 2 ))P ( (W2) + (B/2) - 1 + ' ' ' + (R/2) - I + 1 , 
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j=2 \i=j+l 




1 

Pi 



_ 1 1 \ 1 / 1 1 1 

(n - (R/2))p \R/2 + (R/2) - l + "' + 2 + J + (n - (fl/2))p Vl2/2 + (12/2) - 1 + " ' + 2 



1 / 1 1 \ 1 1 

(n - (fl/2))p {r/2 + (12/2) - 1 ) + (n - (J2/2))p ' 12/2 

n(n — 1). 



+ ••• + 

12 Z? 



(n - (12/2))p (n - (12/2)) 
Therefore, max ueWj? %(u) = #i < n(n - l)R/(n - (R/2)) = 0(n 2 ). 

2) R is odd. Let U Q be a subset of Ur given by U := {u = (x, S + , S-) : x G X Q , S + = S- = 0}, 
where 

X Q := {x : x\ = ■ ■ ■ = xr^i = L + 2, xh+i = L + 1, xh+i , 1 = ■ • • = x n = L}. 

2 2 2 

For the same reason in the preceding case, one can verify that max ue u R Em(u) = max ue ^/ o Em(u). 
Also it turns out that the transition structure, together with the associated transition probabilities, starting 
from U is analogous to that starting from U e . Thus by a similar derivation given above, we can conclude 
again that max ue u R Em(u) < n{n — l)R/(n — (R/2)) = 0(n 2 ). ■ 
Finally, it follows from equations (17)-(19) and Proposition 4 that an upper bound of the mean 
convergence time E qa (xo) of QA algorithm is E qa (xo) < n 2 (n — 1)3(M — m)/2 + n(n — 1)12(12 — 
l)/(n — (R/2)) = 0(n 3 ) for the case 12 > 0. This completes the proof of Theorem 2. 

Remark 3. We have derived an upper bound for the convergence time of QA algorithm on complete 
graphs, by proposing a suitable Lyapunov function for the algorithm and characterizing a Markov chain 
for the state-surplus transition structure. To extend this result to more general topologies, the Lyapunov 
function is still valid (see Remark 2) which in turn validates inequalities (15) and (17). Thus it is crucial to 
establish the relation between graph topologies and the transition structure with associated probabilities, 
as done in the proofs of Propositions 3 and 4 for complete graphs. Establishing such a relation for general 
topologies currently appears to be difficult, but will be explored in our future work. 

V. Numerical Example 

We have proved polynomial upper bounds on the convergence time of QC and QA algorithms for 
complete digraphs. Now we compare these theoretic bounds with numerical simulations, so as to illustrate 
the tightness of our derived results. For this purpose, we consider the following initial states x(0) which 
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Number of Nodes Number of Nodes 

Fig. 10. Convergence time of QC and QA 

5 

x 10 




100 200 300 400 500 600 700 800 900 1000 
Number of Nodes 

Fig. 11. Convergence time comparison among QC, QA, and QA in [7] 

\n/2\ 

correspond to the worst case convergence time: For QC algorithm, we choose x(0) = fl~— — ~T • • • 0] T 

n-2 

(cf. proof of Lemma 3); for QA algorithm, we choose x(0) = [2 1 • • • 1 0] T (cf. proof of Proposition 3). 
The simulation results are displayed in Fig. 10, each plotted value being the mean convergence time of 
100 runs of the corresponding algorithms. 

It is observed that the convergence rate of QC algorithm is approximately quadratic, which demonstrates 
that the derived theoretic bound is relatively tight. On the other hand, the convergence rate of QA algorithm 
appears to be at most quadratic, if not linear. This indicates that the cubic theoretic bound may not be 
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tight, though it is in the same order as the one in [12] also for complete graphs. Thus, deriving tighter 
bounds for the convergence time of QA algorithm awaits future effort. 

Furthermore, we compare the convergence rates of QC, QA, and the original QA algorithm in [7]. 
The results are shown in Fig. 11, each plotted value being the mean convergence time of 100 runs of the 
corresponding algorithms, with the initial states chosen uniformly at random from the interval [—5,5]. 

First it is observed that the convergence rates of QC and QA algorithms (dotted and dashed curves) 
are indeed analogous under the same initial conditions. Also we see that the QA algorithm in this paper 
is considerably faster than that in [7]. This improvement demonstrates that by occasionally requiring 
bidirectional communication, the modifications we have made for QA algorithm effectively accelerate 
convergence. This observation, on the other hand, indicates that there needs extra effort to bound the 
convergence time of the original QA algorithm in [7], which is for average consensus on general digraphs. 
This will be targeted in our future work. 

VI. Conclusions 

In this paper, we have studied convergence time of the quantized gossip algorithms in [7], [8] which 
solve the consensus and averaging problems on digraphs. Specifically, we have derived upper bounds 
- polynomials in the number n of nodes - on the mean convergence time of these algorithms for the 
special case of complete digraphs where the problem becomes tractable. For the consensus algorithm, the 
mean convergence time is 0(n 2 ); this is obtained by bounding the shrinking time of the smallest interval 
containing all states, which results in the special transition structure in Fig. 1. For the averaging algorithm, 
a valid Lyapunov function is proposed and its decay time investigated; this leads us to characterizing 
the convergence time by the hitting time in the Markov chains in Figs. 4 and 5, from which we derive 
0(ra 3 ) time complexity. 

For future work, it would be of ample interest to analyze convergence time of our gossip algorithms 
on more general graph topologies, similar to the work of [13], [16]. A primary difficulty could lie in the 
potentially greater complexity of the state and surplus transition structure, resulting from the topological 
constraints. An alternative might be to explore the relation between the bounds for convergence time and 
the spectral properties of the Laplacian matrix associated to a given topology, as was done in [15]. 
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Appendix 

Proof of Lemma 2. The proof is a direct calculation. By Lemma 1 the mean hitting times of state 
or n satisfy the following linear equations 

E = 0, (20) 

E z = p z E z+ i + r z E z + q z E z -! + 1, z = l,...,n-l, (21) 

E n = 0. (22) 

Since p z = q z , it follows from (21) that p z (E z+1 -E z ) -p z (E z -E z _ 1 ) + l = 0. Let F z+1 := E z+1 -E z . 
Then 

Fz+i = E z . 

Pz 

This is a non-homogeneous first-order linear difference equation, whose solution is of the general form 
To obtain the initial condition F\, consider 



F n + Fn-i + --- + F l = (E n - E n -{) + (S n _i - £ n _ 2 ) + • ■ ■ + {Bk - Eq) = 0, 

n—1 j 



+ F„-i + • • • + F 1 = nF 1 - - 



i , Pi 



From the above we have F\ = (1/n) X)?=i Si=i Vp»- Finally, 



^ = ^ - E = F z + F z -x + • • • + F 2 + Fi 

2-1 j 



j=l i=l 

z n_1 ^ 1 1 
z—l . n— 1 

= (i--)V- + -y^. 

■ 

Proof of Lemma 5. By Lemma 1 the mean hitting times of state n satisfy the following linear equations 

E n = 0, (23) 
Ei = piE 2 + r x E x + 1, (24) 
E z = p z E z+1 + r z E z + g 2 £ 2 _! + 1, z = 2, .., n - 1. (25) 
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Rearrange the terms in (25) to obtain p z (E z+ i — E z ) — q z {E z — E z -\) + 1 = 0. Let F z+ \ := E z+ i — E z 
Then 



F z +\ — — F z , 



Pz Pz 

whose initial condition is F 2 = E 2 — E 1 = — l/pi by (24). This is a non-homogeneous first-order linear 
difference equation with variable coefficients, whose solution is of the general form 



n!) <4>+£ n*H4 



++i 



f o Pi Pl ,r> \i oil Pi 

-—2 / j=2 \i=j+l 



Pi 



Since 



F n + F n _i H h F z+ i — (E n - E n _i) + (E n _i - E n _ 2 ) H h (E z+1 - E z 



— E n — E z — —E z , 



we derive 



n-l 



E z = ~(F n + F n _ 1 + --- + F z+1 ) = 



l=z 



AW pi U\i=Ai Pi J p i 



Finally, 



I n— 1 

Ei = E 2 H = > 



Z=2 



J=2 Pi J Pl j=2 \i=j+l 



1 

+ — . 



Proof of Lemma 6. It follows from Lemma 1 that the mean hitting times of state n satisfy the following 
linear equations 

E T = p 1 E 2 - + r 1 E T + d 1 E l + l, 
Ei = piE 2 + nEi + diEj + 1; 



(26) 



E- — PzE z+1 + riE z + q z E-^—j + c^-E^ + 1, 
^ = PzE z+ i + nE z + q z E z _i + d z £^ + 1; 



(z = 2,...,n-2) 



(27) 



= p n _lS n + r— £— + ^ n _i£^2 + d n-lEn-l + 1, 

-E 'ra-i = r n -iE n ^i + q n ^i E n ^ 2 + d n -iE^i + 1; 



(28) 



#„ = 0. 



(29) 
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Rearrange the terms in (27) as 

Pz(E— - E-) - q z {E, ~ E^) - d z ((E- - E z )) + 1 = 0, 
^ p^E^+i - E z ) - q z (E z - Ez-i) + d z ((E- - E z )) + 1 = 0. 
Let Fjjpf := -%+T ~~ Ez, F z+ \ := E z+ \ — E%, and add these two equations; we obtain 

F- 



; +i + Ez+i — — (F z + F z ) , 



(h ... 2 

Pz ' _/ Pz 

whose initial condition is F^ + Fg= —2/pi by (26). This is again a non-homogeneous first-order linear 
difference equation with variable coefficients, whose solution is 



z+l 



vi=2 



P 



Pi 



j=2 U=j+1 




Now rearrange the terms in (28) 



Vn-i{E n - E-) - q n -i(E— - E—d - d n ^((E— - E^)) + 1 = 0, 
_ - gw _l(.E w _l - £ ra _ 2 ) + d n _i((£— - #n-l)) + 1 = 0. 
Adding these two equations and applying (29), we derive 



J n-1 



Qn-1 
Pn-1 



(F— +F^) + 



Pn-1 



fn-\ 

n 



Pi / Pi 



n— 1 / n—l 

+e n 



Pi 



2 

Pi' 



Vi=2 ^ j=1 \i=j+l 

It is left to obtain the upper bound for E n -i. For this we start by rearranging the terms in (26) as follows: 

(pi + di)% - di^i = pi% + 1, 
_ (p 1 + d 1 )E l -d 1 E T = p 1 E g + l. 
Subtracting the first equation from the second, we have {p\ + 2d\)(E\ — Ej) = p\(Eg — -%). Hence 



Ei — Et 



Pi 



- (Eg — E-^) < Eg — i%. 



1 Pi + 2di 

Similarly, from (27) we obtain a chain of inequalities 

Eg — -% < E3 — E% < ■ ■ ■ < En-2 — E—2 < E n -i — E^z\. 

Finally, rearrange the terms in (28) as 

(Pn-1 + Qn-l + d n -i)E^Y — d n -\ En-i = p n ~\E n + q n -iE-^2 + 1) 
(Qn-i + d n -i) En_i — d n -\E^z\ = q n -i En-i + 1. 
Subtracting the first equation from the second and applying (29), we deduce 

(?n-l + 2dn-l)( En-l - E^i) ~ Pn-lE^ = Qn-lj En-l ~ Ef^) < q n -i( E n -i — E-^j). 

Rearranging these terms we have E n -\ < (1 + (p„_i/d n _i)) E—±. 
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Fig. 12. Idea of induction step 

Proof of Proposition 1. Based on [12, Theorem 2], it suffices to establish the following three conditions: 
(CI) The evolution of (x(k), s(k)), k > 0, is a Markov chain with a finite state space; 
(C2) the set si defined in (10) is an invariant set under QA algorithm; 

(C3) for every (x(0), 0) £ si there is a finite time K a such that Pr [(x(K a ), s(K a )) G si \ (x{0), 0)] > 0. 

For an arbitrary state x(k), observe in QA algorithm that the minimum m{k) is non-decreasing and 
the maximum M(k) non-increasing, where m(k), M(k) are defined in (5). Thus the conditions (CI) 
and (C2) easily follow. It remains to establish (C3) when the digraph Q is complete (Assumption 1), for 
which we proceed by induction on the number n (> 1) of nodes. Let F(k) := M(k) — m(k). Assume 
(x(0),0) £ si; then F(0) > 2. 

(i) Base case: n = 2. Label the two nodes such that cci(0) = m(0) and ^2(0) = M(0). As Q is complete, 
there are two edges, (1, 2) and (2, 1), each of which has a positive probability to be activated. Consider 
the sequence of alternate activation: (1, 2), (2, 1), (1, 2), (2, 1) • • • . Then in QA algorithm, (R3)(i) and 
(R2)(i) will alternately apply, thereby shrinking the interval [m(k),M(k)]. It is easy to see that there 
exist a finite time K a and a positive probability such that x\(K a ) = X2{K a ) = [(xi(0) +X2(0))/2j (thus 
(x(K a ), s(K a )) G si), and at most one node holds a surplus. Also in this process, M(k) decreases by 
at least one and m(k) increases by at least one. 

(ii) Induction step: let r € [2, n — 1]. Suppose that for a network of r nodes, there exist a finite time 
K a and a positive probability such that x\{K a ) = ••• = x r (K a ) = [(l/r) Yli=i X «(0)J, and at most 
r — 1 nodes each holds one surplus. Also suppose that in this process, M(k) decreases by at least one 
and m(k) increases by at least one. 
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Now consider the case with r + 1 nodes. Label them such that m(0) = xi(0) < • • • < x r+ i(0) = M(0). 
In the sequel, we describe a sequence of activating edges, which causes the interval [m(k), M{k)\ to 
shrink, the process being displayed in Fig. 12. The existence of the selected edges follows from that Q 
is complete; and since each edge has a positive probability to be activated, the sequence of activation 
also has a positive probability. 

First, consider the nodes 2, . . . , r + 1. We distinguish three cases as follows. 
Case 1: x r+ i(0) — 2:2(0) > 2. Then applying the hypothesis, we obtain that in a finite time K\ and with 
a positive probability, x 2 {K 1 ) = ■ ■ ■ = x r+1 (Ki) = [(l/r) J2l= 2 X «(°)J- 

Case 2: x r+ i(0) — £2(0) = 1. For each node i (> 2) such that Xj(0) — £2(0) = 1, activate the edge 
(2,i); then (R3)(i) of QA algorithm applies, thereby resulting again in x 2 {K\) = ■■■ = x r+ \{Ki) = 
L(l/r)£S*i(0)J. 

In both cases above, the maximum state decreases as M{K\) < M(0); hence F{K{) < F(0). In 
addition, there are at most r — 1 nodes each having one surplus. Activate (one at a time, in an arbitrary 
order) the edges connecting those nodes with a surplus to the node 1. Thus (R2)(i) applies, and the 
surpluses are consumed to increase x\(k), which in turn causes F(k) to decrease. At time at most 
K[ := K\ + r — 1, all the surpluses in the system can be consumed. 
Case 3: x r +i(0) — x 2 (0) = 0. For this special case, we proceed directly to the next step. 

Second, consider the nodes l,...,r. When F(K[) > 2 (or Case 3 above), applying the hypothe- 
sis we derive that in a finite time K 2 and with a positive probability, x\(K 2 ) = ■■■ = x r (K 2 ) = 
L(V r ) J2l=i x i{K[)\ . Since the minimum state m(k) increases by at least one, we have F(K 2 ) < F(K[). 
Also, at most r — 1 nodes each has one surplus. Select (one at a time, in an arbitrary order) the edges 
connecting the node r + 1 to those with a surplus; then (R2)(i) applies, and the surpluses are consumed. 
Note that, however, here F(k) stays put. At time at most K' 2 := K 2 + r — 1, all the surpluses in the 
system can be consumed. If F(K' 2 ) > 2, we apply the hypothesis again for the nodes 2, ...,r + 1, as is 
done in the first step above. 

Thus we can repeat these two steps, in an alternate fashion, so that F(k) decreases until F{K' a ) = 1, 
for some finite time K' a . There are two possibilities: (1) x\{K' a ) = m{K' a ), others m(K' a ) + l, and at most 
r — 1 nodes each has one surplus; and (2) x r+ i(K' a ) = M(K' a ), others M(K' a ) — l, and at most r — 1 nodes 
each has one surplus. Analogous to the edge activation done above, one can show in both scenarios that 
there exist a finite time K a > K' a and a positive probability such that F{K a ) = 0, and at most r nodes 
each has one surplus. Therefore necessarily, x±(K a ) = ■■■ = x r+ \{K a ) = |_(V( r + 1)) Yll=i x i(®)\- 
Finally, it is evident that in this averaging process, M(k) decreases by at least one and m(k) increases 
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by at least one. This finishes the induction step. ■ 

Proof of Proposition 3. We have given in Section IV-B the proof for the case I = 1, one step away 
from average consensus. It remains to establish (16) for every I € [2, V(0)/2]. Before proceeding, we 
introduce the following notation for an economical representation of the transition structure in Fig. 8: 

([1 1 1 ••■ 1 1 o] T , 1, o) 

t 

([2 11 ■•■ 11 0] T , 0, 0) 

Here ([111 •••11 0] T , 1, 0) represents the upper row of states T, . . . ,n^TT, and ([2 1 1 •••11 0} T , 0, 0) 
represents the lower row of states 1, . . . , n — 1 . It is well to note that the state n (i.e., the average consensus 
state (1, 0, 0)) is not involved. Observe that only the triples in are used, and only the triple with positive 
surplus S+ > has a transition probability to the average consensus state. We will use this notation to 
display the transition structures in the subsequent analysis. 

(i) Two steps away: from U 2 to U\. The corresponding transition structure is displayed in Fig. 13; 
there are four triples, representing four rows similar to the above. These rows can be arranged into three 
blocks B\, B 2 , and B3 as shown. Notice that the displayed triples are all in U®, and only those triples 
with positive surplus S+ > have a transition probability to U\. One can readily see that starting from 
the triple ([3 1 1 • • • 1 1 — 1] T , 0, 0), the mean hitting time of U\ is the longest; thus we need to analyze 
the whole structure. 

In the sequel, the structure will be simplified in two steps. First, treat the transition to £>3 as a selfioop at 
the triple ([2 1 1 • • • 1 1 - 1] T , 1, 0) in B 2 . This modification increases the mean hitting time starting from 
B\. To see this, note that the triple in B3 has more positive surplus S+, which results in higher probabilities 
of moving towards U\ . It then follows that selfiooping in B2 takes longer time to hit U\ than transiting to 
B 3 . Second, combine ([3 1 1 • • • 1 1 - 1] T , 0, 0) in B x and ([2 2 1 • ■ ■ 1 1 - 1] T , 0, 0) in B 2 . This amounts 
to combining the corresponding two rows of triples. It can be verified that the associated transition 
probabilities in these two rows are the same, except for those moving to ([2 1 1 • • • 1 1 — 1] T , 1,0). 
Since the latter means moving towards U\, taking the smaller transition probabilities from the two rows 
will increases the mean hitting time. 

After the above modifications, the transition structure is simplified to the one displayed in Fig. 5, with 
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Fig. 13. Two steps away: from U2 to U\, 




([2 11 ••■ 11 -2] T , 2, 0) I 

u 

([2 2 1 ■ ■ ■ 11 - 2] T , 1, 0) 

ti 

([2 2 2 ■ ■ ■ 1 1 - 2] T , 0, 0) 
S3 




Fig. 14. Three steps away: from U$ to i/2. 



the following transition probabilities: 

pi = (n - 2)p, ^ = p; p„_i = p, g n _i = (n - 2)p, d„_i = (n - 2)p; 
p z = (n - 1 - z)zp, q z = (z-l)p, d z = (z-l)p (z = 2, ...,n - 2). 

Hence, we have max ue ^o < -En-i, where i£ n _i is given in (4). Invoke the formulas in Lemma 6, 

and perform an analogous calculation as before; we then obtain that max ug ^o E\(u) = 0(n 2 ). 

(ii) Three steps away: from W3 to U2- The corresponding transition structure is displayed in Fig. 14; 
we now have four blocks. Since starting from the triple ([4 1 1 • • • 1 1 — 2] T , 0, 0) the mean hitting time 
of IA2 is the longest, we need to analyze again the whole structure. 

We take three steps to simplify the structure. First, treat the transition to £>4 as a selfloop at the 
triple ([2 1 1 • • • 1 1 — 2] T , 2, 0) in B^. This is the same as that in (ii), and hence increases the mean 
hitting time starting from B\. Second, treat the transitions to block B3 as selfloops at the corresponding 
triples in £>2- This modification also increases the mean hitting time. To see this, compare the structure 
of B2 and its counterpart in B3 (i.e., the lower two triples alone). One may verify that the former has 
longer rows of triples and higher probabilities of moving away from U2- Hence, the mean time taken 
to hit U2 in the structure of B2 is longer than that in its counterpart in B3. Further, the top triple 
([2 1 1- ■ ■ 1 1 — 2] T ,2,0) in B3, with more positive surplus S + , makes the mean hitting time even 
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shorter. Therefore, selfiooping in B2 increases the mean time to hit U2 compared to transiting to B3. 
Lastly, combine ([4 1 1 • • • 1 1 - 2] T ', 0, 0) in Bi and ([3 2 1 • • • 1 1 - 2] T ', 0, 0) in B 2 , as is done in (ii). 

The above simplifications lead us again to the structure displayed in Fig. 5, with exactly the same 
transition probabilities as (ii). We thus obtain max ugW o E\{u) < E n -\ = 0{n 2 ). 

(iii) General I (> 3) steps away: from Ui to Ui-\. The corresponding transition structure consists 
of I + 1 blocks. Apply an analogous procedure to simplify this structure; it can be found by a similar 
argument that transiting to further blocks will accelerate hitting U\-\. Consequently, the structure with 
l + l blocks can also be reduced to the one in Fig. 5, the probabilities of which are those in (ii). Therefore, 
max ueU o E} (u) < £ n _i = <3(n 2 ). ■ 



